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The invention relates to a method of segmenting a three-dimensional 
structure from a three-dimensional data record, containing in particular medical data values, 
using a deformable model. The surface area of the deformable model is formed by a 
network of meshes, the meshes connecting points of the network on the surface area of the 
10 model. The invention further relates to an image processing device for performing this 
method and to a computer program for controlling the image processing device. 

Amemodofmeldndmentionedatme beginning is known from an essayby 
Mclnerney et al. "Deformable models in medical image analysis: a survey" in the journal 
Medical Image Analysis, 1 (2):91-108, 1996. According to that article, a deformable model 
15 is represented as an elastic surface area which can, for example, change its shape and 
position by minimising the weighted sum of an internal and an external energy as they are 
called. The internal energy in this context can be of any magnitude, which is the smaller, 
the less the model is deformed, while the external energy can be of any magnitude, which 
becomes smaller as the deformable model approaches so-called candidate points, which are 
20 probably arranged on the surface area of the structure to be segmented. Among a 
predetermined number of points in the three-dimensional data record, a candidate point 
may,for example, bethatpoint for which the probabihtyofits being located on the surface 
area of the structure to be segmented is greatest. A measure of this probability is the 
structure function F(x). The greater the value of the structure function F(x), the greater is 
25 the probability that the point x in the three-dimensional data record is located on a surface 
area of the structure to be segmented. 

In known methods, the meshes take the shape of a triangle, for example, the 
comers of which are defined by three adjacent points of the network on the surface area of 
the model. Along normal plane lines of the triangle plane candidate points are sought in the 
30 three-dimensional data record for which the structure function exhibits a maximum. In this 
context, the function F(x) can, for example, be defined by the gradient of the data values at 



WO 2004/063987 PCT/IB2003/006255 



position x projected onto the normal plane line. 

Once candidate points have been determined for the meshes of the network 
defining the surface area of the model, the points of the network of the model are 
recalculated on the basis of the candidate points found. In the process, the points of the 
5 network are calculated in such a way that for the newly found points of the network, the 
weighted sum of internal and external energy reaches a minimum. After that, the method is 
repeated several times, on the basis of the newly calculated points of the network, the 
model coming closer to the shape of the structure to be segmented with each repetition. 
This iterative method is completed as soon as a termination criterion, e.g. a specified 

1 0 number of repetitions, has been met. 

One disadvantage of this method is that a structure function for the entire 
deformable model is used, so that structures whose surface areas have varying 
characteristics can only be segmented inadequately. This varying characteristic might, for 
example, be a gradient of the data values, which points in opposite directions for different 

15 parts of the surface area. In other known methods, each mesh or each triangle plane is 
allocated a structure function, which leads to a high level of computation effort. Apart from 
that, segmenting errors often occur with this method, since whenever there is a small 
deviation between the deformable model and the structure to be segmented, it is no longer 
possible to assign each structure function to the corresponding surface region of the 

20 structure to be segmented. 

It is the object of the present invention to provide a method in which these 
segmenting problems are less pronounced. This object is achieved by a method of 
segmenting a three-dimensional structure from a three-dimensional data record, containing 
in particular medical data values, comprising the steps of: 
25 a ) providing a three-dimensional, deformable model whose surface area is 

formed by a network of meshes, which connect together points of the network on the 
surface area of the model, the meshes being divided into groups and each group being 
allocated a structure function, 

b ) arranging the model at the position in the three-dimensional data record in 
30 which the structure to be segmented is located, 

c) determining one candidate point for each mesh with the aid of the structure 
function of the group to which the mesh concerned belongs, 
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d) recalculating the points of the network of the model on the basis of the 
candidate points found, 

e) repeating steps c) and d), taking the newly calculated points of the network 
into account, as long as no termination criterion is met. 

Unlike known methods, the meshes of a model according to the invention 
are divided into groups, and each group is assigned a structure function. This makes it 
possibletodetermmecandid^^ 

on a surface area of the structure to be segmented, even in the case of a surface area which 
vanes considerably in its characteristics, which leads to improved segmenting. The varying 
characteristic can, for example, be the direction of a gradient of the data values, as already 
mentioned above. Furthermore, this method requires less computation effort compared to 
assigning one structure function to each mesh. 

In the embodiments according to Claims 2 and 3, the meshes of a group have 
similar characteristics. Since similar characteristics in the meshes are due to similar 
15 characteristics of the corresponding surface regions of the structure to be segmented 
forming groups in this way leads to a differentiation between different parts of the 
structure, e.g. the different parts of an organ or bone. 

Claims 4 and 5 define preferred methods of determining candidate points. 
Claim 6 defines a structure function, and Claim 7 describes a preferred 
20 method of recalculating the points of the network, which leads to good segmenting results. 

An image processing device for carrying out the method of the invention is 
described in Claim 8. Claim 9 defines a computer program for controlling the image 
processing device of Claim 8. 

The invention will now be explained in more detail with reference to the 

25 drawings. 



Fig. 1 shows an image processing device suitable for carrying out the 

method of the invention, in a schematic representation, 

30 Fi & 2 sh °ws a flow chart of the method of the invention, 

Fig. 3 shows a flow chart of a method for dividing meshes of a deformable 
model into groups, 
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Fig. 4 shows a perspective view of a deformable model of a left heart 

ventricle, 

Fig. 5 shows a perspective view of a femur model, and 

Fig. 6 shows two distributions of brightness values for the average data 
5 vectors of two groups, 



The image processing device showninFig. 1 comprises an image processing 
and control processor 1 with a memory 2, in which a deformable model ofa structure to be 
10 segmented and a three-dimensional data record, containing in particular medical data 
values, can be stored. The image processing and control processor 1 can be coupled to an 
imaging device, which is not shown in more detail, such as a magnetic resonance or 
computer tomography device, e.g. by means of a glass fibre cable 3. The structure 
segmentedbythe image processing and control processor 1 can be reproduced on a monitor 
15 4. The user can access the image processing and control processor 1 via a keyboard 5 or 
other input devices, which are not shown in more detail in Fig. 1 . 

Fig. 2 shows the sequence of a segmenting process which can be performed 
with the image processing device according to Fig. 1 . 

After initialisation in step 100, the deformable model ofa structure to be 
20 segmented - in this working embodiment, that ofa left heart ventricle - is provided in step 
102. The surface area of the model in this connection is formed by a network of meshes 
which connect the points of the network on the surface area of the model. The meshes are 
in the shape of triangles, the comers of which are in each case defined by three adjacent 
points of the network on the surface area of the model. Instead of a model triangulated in 
25 this way, the surface area could also be defined by any polygonal structure in which the 
meshes of the network are not triangular, but which connect a number of points of the 
network, other than three, forming a polygon. 

A deformable model can be generated from a plurality of framing data 
records in each of which the same object has been segmented manually by a skilled person, 
30 and the surface areas of the segmented objects are represented by networks of the same' 
topology. The same topology in this context means that each segmented object has the 
samenumberofmeshesandmatmesamemeshhasmesamenimiberofadjacentmeshesin 
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different training data records. If, therefore, the meshes m each data record are numbered 
and, forexample,inthe first data record mesh no. 1 has the adjacent meshes no. 2 to no 9 
meshno. 1 is also adjacent to meshes no. 2 to no. 9 in all the other data records. Averaging' 
m the simplest case arithmetical averaging, over all the segmented objects leads to an' 
object that is referred to as a deformable model. 

The method of the invention can also be applied if the deformable model has 
been generated in a different manner in other embodiments, e.g. by the simple triangulation 
of a model of a left ventricle. One triangulation method which is frequently used is known 
from the Article "Marching cubes: A high resolution 3d surface area construction 
algorithm", W. E. Lorensen and H. E. Cline, Computer Graphics, 21(3): 163-169, 1987. 

The meshes of the deformable model are now divided up into groups in step 
1 04, it being possible to perform the method of the invention with any method of division 
into groups. 

One possible method of division into groups is shown in a first embodiment 
15 m Fig. 3. First of all, in step 200, N data records are generated, the data values of which 
each represent a left ventricle in this working embodiment. In step 202, the deformable 
model is then adapted to the ventricle in each data record in accordance with the known 
method which was described at the beginning, minimising the weighted sum of the internal 
and external energy, which leads to iVmodels of the ventricle, all of which have the same 
20 topology. 

hi step 204, anumber Gof groups is selected, into which the triangle planes 
are divided. If the surface area of the structure to be segmented has different regions, which 
clearly differ from one another in the three-dimensional data record, the number G is 
preferably selected such that it is equal to the number of different surface area regions This 
clear difference might, for example, be a gradient of the data values, which, for one surface 
region, is directed into the interior of the structure to be segmented, and, for another surface 
region, points to the outside. It is known that precisely these different gradient directions 
can occur with the left ventricle, so that in this working embodiment G = 2 is selected. 

In the subsequent step 206, a feature is assigned to each triangle plane of 
each model. This feature is a data vector, which contains the data values (in this case- 
hngbtness values) located on the normal of the triangle plane. The normal in this case runs 
through the centralpoint of the triangle plane, and the data vector consists, for example of 
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10 data values located on to normal on one side of to triang.e p!ane, one data value 
located on to triangle pkute, and a further 10 data values located on the other aide of or 

to^W^Plane.e.g.togradientoftodatevaluesintoposinonoftocentopoint 
of to area projected onto to norma!. More than one feature per triage P ,ane is also 
possible. 

An average feature of each group is detennined in step 208. For Urispurpose 
first of all a feature space containing all to values which features that have heen assigned 
to to tnangle planes can assume is divided into G sub-space, Then, an average value of 
to features is determined for each sub-space, and each average value of a sub-space is 
defmed as a gmup feature, m mis working embodiment, to feature space consist of to 
components of to data vectors, where each component can assume values between 0 and 
51 1. The featore space is now subdivided into two sub-spaces. The features of to first 

s P-^^vec to rswh„se«,n 1 ponen te havevaluesbe to een0and255,andtofeamre S 
1 5 of to second space are data vectors whose components have values between 256 and 5 1 1 

Arithmetical averaging produces, as to average feature of to first sub-space, a data vector 
theetnnponentaofwmchhavetovahre 127.5. The average data vector of to second sub- 
space accordingly has to value 383.5. Thus to average feature of to first group i e to 
group feature, is to data vector whose components have to value 127.5, and to 
20 corresponding feature of to second group is to data vector with to values 383.5. 

mste P 210,toas S ignedfeaturesofaUtotriangleplanes are compared with 
afi to group features with to aid of a similarity criterion. Each Wangle plane is ton 
asstgned to to group whose group feature is most similar to to feature of to Wangle 
Plane according to to similarity criterion. The similarity criterion here can be any 
25 express™, which compares features of to triangle planes or of the groups. This can for 
example, beacorrelationofthedatavectorsoftwotriangleplanestobe compared or Iain 
tins working embodiment, the norm of the difference between two data vectors. 

After each triangle plane has been assigned to a group, in step 212 a new 
average feature is detennined for each group, i.e. a new group feature in each case. For this 

30 P^toaverageoftoassrgnedfeaturesofaUtoWangleamagroupisdetorminedin 
fins working embodiment, to average of to dato vectors of to triangles is detennined 
anthmetically. The result of to arithmetical averaging is to new group feature 



WO 2004/063987 PCTOBMOJAHKaM 

7 

to step214, there isacheck on whemeratemunation criterion has beenmet 
Thts tennuration criterion can, for example, be a predetermined number of iterations or a 
change in the group feanues ma, is so small as to be hardly present. If the termination 
cntenon ,s met, the process continues with step 216. Omerwise, the method is continued 
5 with step 210. 

Since memodels in the different datorecordshavemesametopology every 
^'-n^attan^epianewimmesameneighbonrhood^ationship, Triangles with 
mesameneighbonAoodrelationsmpscanbec^ntamedmaplurabtyofgxnups 
be present more man once in one group, to step 216, these Wangles are assigned to the 
10 group m which these triangles are contained most frequently. 

In step 218, a deformable model is generated from the N models by 
averaging, by determining the arithmetical average of, for example, depositions of all the 
tnangle planes which have the same neighbourhood relationships. 

This method is described in detail in "Pattern Classification and Scene 
15 Analysis", Duda and Hart, Wiley, 1973, to which reference is made here. 

The resulting first group 10, the second group 12 and the group border 14 of 
the deformable model M of fhe left ventricle are depicted in Fig. 4. The tiiang.es which 
form Ore first group 10 in fids model correspond to the region of the .eft ventricle which 
touches fire right ventricle. La. the group assignment has made i, possible to identify 
20 differenfparts of me heart. This identification of individual re^ons of tire stiucture ,o be 
segmented is always possible whenever tire characteristics of the structure detectable in tire 
data record which distinguish the individual regions of the stiucture from each other are 
used as features for comparison by means of a similarity criterion. Identifying individual 
regions of me structure to be segmented by meam, of the group assignment makes it 
25 possrble to extract these regions, so that they can be further processed separately from one 
another. 

A further example of this identification is shown in Fig. 5. That shows a 
fnangulated model of a femur F, tire triangle planes of which have been subdivided into 
four groups 20, 22, 24, 26 by means of fire method of the invention described above Her. 
30 too, parts of the stiucture to be segmented have been identified by tire group assizer* 
Thus group 22 defines the head of tire femur, and group 26 the greater trochanter. 

By selecting the features to be compared and the similarity criterion 
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appropriately, it is thus possible to identify different parts of a structure to be segmented, 
e.g. of an organ or bone, which can then be extracted from the data record and processed 
individually. 

In a different embodiment, in order to divide the triangle planes of the 
deformable model into groups, first of all a triangle is selected at random and its feature is 
compared with the feature of an adjacent, neighbouring triangle. Features in this case are 
the data vectors described above. The method can, however, also be carried out with other 
features. The comparison is made between the triangle planes, using the similarity criterion 
which has likewise been explained above, two triangle planes being regarded as similar if 
the similarity criterion is higher or lower than a predetermined threshold value. This might, 
for example, be when the threshold value of 0.9 is exceeded in the correlation of two data 
vectors or when the result falls short of the threshold value of 0.1 in the norm of the 
difference between two data vectors. If the two triangles are similar, the two triangles form 
a first group and one or more group features are determined by averaging the individual 
features of the triangles. In this working embodiment, the individual components of the 
data vectors could, for example, be averaged arithmetically, so that an average data vector 
is generated for that group. Then the same procedure is repeated with a further adjacent 
triangle, the data vector of that triangle being compared with the average data vector of the 
group. This method is continued until none of the triangles adjacent to the triangles of the 
group is similar to the average data vector of the group any longer. Then a further triangle 
which does not belong to any group is selected, with which the method described above is 
repeated, forming a second group. This procedure is repeated until every triangle has been 
assigned to a group. 

A further embodiment of the group assignment procedure is manual 
assignment by an expert, who groups together the meshes which are similar with regard to 
one or more features of the structure to be segmented. 

In step 106, a structure function is assigned to each group, a structure 
function being any function that furnishes a large value whenever there is a high degree of 
probability that a candidate point is located on the surface area of the structure to be 
segmented. Since different regions of the surface area of the structure to be segmented can 
have different characteristics, it is a good idea for these regions each to define a separate 
structure function. 
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A candidate point can define a candidate triangle plane. In this case the 
candtdat. triangle p.ane is a triangle p.ane which has been displaced for the purpose, of 
adaptation 'o"«™ou 1 retob.seg m en,e4inthedirecfi„nof,heno m a.ofa W angleplane 
^ the normal intersect. 

In order to determine the structure funcfions, the average of the features of 
fte ta an g leplanesofagroupi 5 de,erm in e4ix.tafifi swor&gemb( ^ entthe f 
file corresponding data vectors, as already described above, if fins has no, already been 
done. The resulting group feature or fire average data vector defines the structure Amotion 
ofthe group. The average data vector 30 of group !0 and fire average data vector 32 of 
^oup 12 is shown in Fig. 6. There, the brightness value has been plotted on fire vertical 
axts, and an index has been plotted on the horizonta! axis, which counts the date vahtes 
along the normals. 

The sfiucture ftnction in fins working embodiment is a measure of the 

15 ^'-Wchafeatoreofacanmda.epom.ormecanmdateniang.ep.anedefinedby.he 
candidate point is identical to the cotresponding group feature; fire similarity criterion 
descnbed above can be used as a measure ofthe degree to which they are identical. Thus 
fire structure faction can, for example, be the reciprocal of fire normal ofthe difference 
between the date vector of a candidate triangle plane and the date vector of fire 

20 corresponding group. 

For each mesh of fire deformable model, a confidence is determined with a 

confidencecriterioninsteplOS.Thecnfidenceofameshsteteshowwellam^hmatohcs 
tts tespecfive grcup. When the confidence is high, mere is great certeinty that the mesh 
concenter! really could be aligned to fire group concerned, because of its features When 
the confidence is low, it is less certain that i, belongs to a particular group. 

Onepossibility of indicating a confidence criterion is the similaritycriterion 
described above. If the features of a triangle and a group are very similar according to fire 

thus be used directly as the confidence. 

A further confidence criterion is finished by fire group assignment shown in 

F*3.mstep2 1 6mere,for«riang 1 ep I a„e 5 wimmesameneighbomhoodrete«onship S or 
alter the average has been found for those triangle planes in step 218, for every triangle 
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plane, the frequency with which it belongs to the group is determined. The greater that 
frequency is for a Mangle plane, the better the features of that triangle plane match the 
group feature, The frequency of belonging to the group can thus be used as confidence 
The deformable model of the left ventricle will be used in the following to 
5 segment a left ventricle in a three-dimensional data record automatically. 

For this purpose, in step 110, the deformable model is first of all aligned 
relative to the structure to be segmented in such a way mat the surface area of the model 
corresponds as well as possible to the contours of the structure to be segmented. 

Instep 112, a candidate point i, is determined for each triangle plane on the 
10 -^-eaofthedeformablemoda 

plane lines of the triangle and on a candidate triangle plane. The position of the candidate 
point x, on the normal plane lines is now selected such that the structure fimction F j( ± t) 
of the group to which the triangle plane concerned belongs exhibits a maximum^ The 
indices i and J here indicate different candidate points or groups. 

After a candidate point has been found for each of the triangle planes on the 
surface -a ofme deformable model, m step 114 me surface area of me deformable 
is adapted to the candidate points found. To do this, the model is deformed in such a way 
that the so-called energy function 

E = E i*+<*E^ (1) 
20 assumes a minimum. Here, E ext is the- external energy which moves the 

tnangle planes in the direction of the candidate point, determined in step 1 12, while E int is 
the mtemal energy wMch opposes any d^^ 

is typically selected such that the two energies contribute to the total energy in equal 
measure, 

25 The external energy chosen can be any parameter that becomes smaller as 

the deformable model approaches the structure defined by the candidate point, In this 
workmg embodiment, the external energy E ext is described by the following equation: 

^«t=2>,(X,-S,) 2 , (2 ) 

where all the triangles are summed. The weighting factor W , is calculated 
30 according to the following equation: 

w^maxfO^Cx,)}. (3) 
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AccordingtoequationOXcandidatepoints i, whose features correlate well 
other candidate points. 

mother embodiments, me weighting factum „, could also take into acconnt 
5 fte^dencedetenmnedinstep lOS-Huscandidatepointswhichexhibitatowerdegree 

greater degree of confidence has heen determined. I, would also be possible to take into 
accountthe fiuctoation margin of .he group concerned. Candidate points in a greup with a 
large fluctuation margin, , e . in which me features of a number of Wangle p.anes diverge 
10 consmerably fiom me average feature of me greup, ought to be token into account with a 
^-^'tacanmdatopo^ma^fi.a,^,^^^^ 

^""^^^^"^yparameterthatbecomessmaUer.theless 
to the relation 6 
15 V here is the number of triangle planes in the model,, a scaling factor Ra 

m^onnutrix.and^mematoucebebveen.wocen.repotooftwoWaugleptoesom^ 

~^^»^tIl 1 es«N(Ocom^meoen«repom b ofuto S e« m g,.p 1 an M 
Wb < h -^™e«anglep M 

to equation (1) is minimised for the first time, s = l, and R is the unit matrix 

20 k , AftertheCen,rep0intSIi0fthetri ^Planeaofmedefonnedmode.have 
^^-^-^smgmeene^macc^withe,^^^ 

actorsandanewrotofionmaWxRareda.enimieilTodo.ms.ttecurrentc^pomtox 
-^^wandmeac^^^^^^^^^^ 

interna, energy becomes minimal, hi other embodiments, the centre points x„ me scafing 

^torandme^afionmattxvariableasweUwhendeterminhigmenmtoumofuietoto, 
energy E. 

AAerthedefornmblemodelhas,inst=p 114, been adapted to toe structure to 
30 ^"^^^ whether a. erminafion criterion has been 

;^~ On ^ 0nCm - f0r ^ ,e ' beaPr ^~ ^^repetitions 
ofstopsn2andll4o I achangemmep MiaonofthepointsofthenetworkthaUsso 
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as to be hardly present. If the termination criterion is met, the segmenting procedure 
terminates in step 1 18. If, on the other hand, the termination criterion is not yet met, the 
method continues with step 112. 

The model obtained according to the segmenting procedure can be stored 
5 and displayed on the monitor. It represents the segmenting of the structure to be segmented 
contained in the three-dimensional data record. 

After steps 100 to 108 have been carried out once, one has a deformable 
model with meshes, which have been subdivided into groups, and with structure functions 
that have been assigned to the groups. For this reason, when segmenting further structures, 
10 it is possible to refer to this model, so that steps 104 to 108 can be dispensed with. 



